Conventionally, there is an image generating apparatus to generate a panoramic image from images captured by a plurality of cameras. For example, the image generating apparatus acquires surrounding images from a 360-degree camera in which a plurality of cameras is provided to cover all directions of a shooting range. The image generating apparatus projects a plurality of acquired camera images on a virtual spherical screen that centers on the 360-degree camera. The image generating apparatus generates a spherical image by blending pixel values of overlap portions where part of each camera image overlaps each other on the spherical screen. Next, the image generating apparatus converts the spherical image projected on the spherical screen into a planar image by fluoroscopy projection in which the point of sight is the center of a sphere, cylindrical projection in which the medial axis passes the center of a sphere, or polar coordinates projection in which the centers are coincident, and generates a 360-degree panoramic image.
Moreover, there is a technology which reduces influence of parallax on generation of a panoramic image. Parallax is a difference in angle of sight at which a subject is viewed, which is caused due to a difference between positions at which a plurality of cameras is installed. This technology reduces the influence of the parallax by using the fact that “when the distance from the camera to a subject is large enough compared with the distance between cameras, the distance between cameras can be neglected”. For example, when the distance from the cameras to the subject is far enough, since the radius of a virtual spherical screen on which camera images are projected can be set to be infinite, a panoramic image is generated while suppressing the influence of the parallax. As for related arts, refer to the following patent documents: Japanese Laid-open Patent Publication No. 2005-056295; Japanese Laid-open Patent Publication No. 2008-77628; Japanese Laid-open Patent Publication No. 2006-054662; and Japanese Laid-open Patent Publication No. 2000-278602, and the following Non-Patent document: S. Ikeda and two others, “Generating a Panorama Movie Using an Omni-directional Multi-camera System”, Institute of Electronics, Information and Communication Engineers, 2002.
Incidentally, in the conventionally-known image generating apparatus described above, a panoramic image, in which the overlapping portions of each camera image match each other, is generated when there is no parallax between the camera images. Therefore, in connection with the conventional image generating apparatus, the cameras need to be installed sufficiently adjacent to each other to the extent that the parallax can be neglected, or the cameras need to take an image of only a subject which is sufficiently far from the cameras to the extent that the parallax can be neglected.
Here, it might be useful to generate a panoramic image from camera images captured by a plurality of cameras installed at different positions and provide it to the user. For example, it is considered to be useful at the time lane-changing or parking if camera images are acquired from in-vehicle cameras mounted at the front, rear, left and right side of a vehicle, and a panoramic image is generated from the acquired camera images and provided to the driver. However, the conventional image generating apparatus had a problem of generating a panoramic image giving a sense of unnaturalness when generating a panoramic image from a plurality of camera images with parallaxes because the apparatus double-projects the overlapping portions of each camera image on the spherical screen.
The reason why the conventional image generating apparatus generates the unnatural panoramic image is described referring to FIG. 29 and FIG. 30. FIGS. 29 and 30 are diagrams to describe the problem of the related art. FIG. 29 illustrates an appearance of a vehicle 10 seen from the side, and FIG. 30 illustrates an appearance of the vehicle 10 seen from above. In examples illustrated in FIGS. 29 and 30, the vehicle 10 includes an image generating apparatus mounted thereto. The vehicle 10 also includes a camera 20 mounted at the front side, a camera 21 mounted at the rear side, a camera 22 mounted at the left side, and a camera 23 mounted at the right side. Each of the cameras 20 to 23 captures its own camera image around the vehicle and the image generating apparatus mounted to the vehicle 10 projects the captured camera images on a virtual spherical screen 50. The spherical screen 50 is set, centering on the vehicle 10.
Referring to an example of FIG. 29, a case where a subject 30 is located in the distance is described. The image generating apparatus acquires camera images, in which the subject 30 is a transmitter located in the distance, from the cameras 21 and 22. The image generating apparatus projects an image corresponding to a position “P” within the subject 30, which is captured by the camera 21, on a position “P1” of the spherical screen 50. On the other hand, the image generating apparatus projects an image corresponding to the position “P” within the subject 30, which is captured by the camera 22, on a position “P2” of the spherical screen 50. In other words, the image generating apparatus doubly projects the images corresponding to the position “P” within the subject 30, on the position “P1” and the position “P2”. Therefore, the image generating apparatus generates a panoramic image which gives the driver a sense of unnaturalness for the result of converting the doubly projected spherical image into a planar image.
Referring to an example illustrated in FIG. 30, a case where the subject 30 is a nearby object is described. The image generating apparatus acquires camera images, in which the subject 30 is a white line on a road, from the cameras 20 and 23. The image generating apparatus projects an image corresponding to a position “P” within the subject 30, which is captured by the camera 20, on a position “P1” of the spherical screen 50. On the other hand, the image generating apparatus projects an image corresponding to the position “P” within the subject 30, which is captured by the camera 23, on a position “P2” of the spherical screen 50. In other words, the image generating apparatus doubly projects the image corresponding to the position “P” within the subject 30, on the position “P1” and the position “P2”. Therefore, the image generating apparatus generates a panoramic image which gives the driver a sense of unnaturalness for the result of converting the doubly projected spherical image into a planar image.
Moreover, even if the conventional image generating apparatus uses the technology which reduces the influence of the parallax, the conventional image generating apparatus generates an unnatural panoramic image from camera images with parallax. This is because the distance between cameras cannot be neglected when the subject exists near the cameras. For example, even if the radius of the spherical screen, onto which the image corresponding to the position “P” within the subject 30 is projected, is set to be infinite, the distance between the position “P1” and the position “P2” on a projection surface are not converge, and as a result, an unnatural panoramic image is generated.